Short-time stopping detection from red light camera evidentiary photos

ABSTRACT

A method for detecting a vehicle running a stop signal includes acquiring at least two evidentiary images of a candidate violating vehicle captured from at least one camera monitoring an intersection. The method includes extracting feature points in each of the at least two evidentiary images. The method includes computing feature descriptors for each of the extracted feature points. The method includes determining a correspondence between feature points having matching feature descriptors at different locations in the at least two evidentiary images. The method includes extracting at least one attribute for each correspondence. The method includes determining if the candidate violating vehicle is in violation of running the stop signal using the extracted attribute.

BACKGROUND

The present disclosure relates to a vision-based system and method formaking a traffic regulation violation decision regarding vehicleactivity during a stop light signal. The system automatically processesevidentiary images to discriminate between violating vehicles that run ared light and/or stop signal and non-violating vehicles that abruptlystop. However, the present disclosure is amenable to other likeapplications.

Red Light Camera Systems (RLCS) are traffic regulation enforcementsystems that detect and identify vehicles that enter an intersectionagainst a red traffic light and, therefore, are in violation of atraffic regulation. These systems can detect the violating vehicles byidentifying license plate numbers and/or the make and model of thevehicles from photographs captured by red light cameras. A citation isthen issued to the owner of the vehicle identified in a photograph.

In more specific detail, FIG. 1 shows how an existing RLCS systemoperates in the PRIOR ART. An enforcement camera 10 is installed in aprotective metal box attached to a pole 12 at an intersection. Tomeasure vehicle speed, two closely spaced induction loops (“sensors 14A,14B”) are embedded in the pavement near the stop line 16. When a vehicleactivates both sensors within a predetermined time threshold, thesensors trigger the cameras 10, 18 to capture the event as a series ofphotographs or a video clip, which shows the vehicle 20 as it enters andproceeds through the intersection on a red light signal 22.

Existing systems generate a number of false detections, which can resultin the issuance of erroneous tickets. These false detections mainlyresult from vehicles that abruptly stop at the stop line aftertriggering the sensors 14A, 14B within the predetermined time threshold.These systems furthermore require a law enforcement official review thephotographs to determine if a violation occurred. This time consumingtask also results in a significant number of the violations beingrejected as false detections.

An improved system and method is desired which automatically detectsnon-violating vehicles, which are falsely identified as violatingvehicles in the existing system. A system and method are desired whichdiscriminates between violating and non-violating vehicles usingevidentiary images.

INCORPORATION BY REFERENCE

-   David G. Lowe, et al., Distinctive Image Features from    Scale-Invariant Keypoints, International Journal of Computer Vision    60.2 (2004) at pg. 91-110 is fully incorporated herein.-   Herbert Bay, et al., Speeded-up Robust Features (SURF), Computer    Vision and Image Understanding 100.3 (2008) at pg. 346-359 is fully    incorporated herein.-   Chris Harris and Mike Stephens, A Combined Corner and Edge Detector,    Alvey Vision Conference, Vol. 15, 1998 is fully incorporated herein.-   Edward Rosten and Tom Drummond, Machine Learning For High-Speed    Corner Detection, Computer Vision-ECCV (2006) at pg. 430-443 is    fully incorporated herein.-   Carlo Tomasi and Takeo Kanade, Detection and Tracking of Point    Features, Technical Report. CMU-CS-91-132m School of Computer    Science, Carnegie Mellon Univ. (1991) is fully incorporated herein.-   P-E Forssen and David G. Low, Shape Descriptors for Maximally Stable    Extremal Regions, International Conference on Computer Vision    IEEE (2007) at pg. 1-8 is fully incorporated herein.-   Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for    Human Detection, Computer Vision and Pattern Recognition (2005) is    fully incorporated herein.-   Yan Ke and Rahul Sukthankar, PCA-SIFT: A More Distinctive    Representation For Local Image Descriptors, Computer Vision and    Pattern Recognition (2004) is incorporated fully herein.

BRIEF DESCRIPTION

One embodiment of the disclosure relates to a method for detecting avehicle running a stop signal. The method includes acquiring at leasttwo evidentiary images of a candidate violating vehicle captured from atleast one camera monitoring an intersection. The method includesextracting feature points in each of the at least two evidentiaryimages. The method includes computing feature descriptors for each ofthe extracted feature points. The method includes determining acorrespondence between feature points having matching featuredescriptors at different locations in the at least two evidentiaryimages. The method includes extracting at least one attribute for eachcorrespondence. The method includes determining if the candidateviolating vehicle is in violation of running the stop signal using theextracted attribute.

Another embodiment of the disclosure relates to a system for detecting avehicle running a stop signal. The system comprises a traffic regulationenforcement device including a memory and a processor in communicationwith the processor. The processor is configured to acquire at least twoevidentiary images of a candidate violating vehicle captured from atleast one camera monitoring an intersection. The processor is configuredto extract feature points in each of the at least two evidentiaryimages. The processor is configured to compute feature descriptors foreach of the extracted feature points. The processor is configured todetermine a correspondence between feature points having matchingfeature descriptors at different locations in the at least twoevidentiary images. The processor is configured to extract at least oneattribute for each correspondence. The processor is configure todetermine if the candidate violating vehicle is in violation of runningthe stop signal using the extracted attribute.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows how an existing RCLS system operates in the PRIOR ART.

FIG. 2 is an overview of the present method.

FIG. 3 is a schematic illustration of a vision-based system for making atraffic regulation violation decision regarding a vehicle running a stoplight.

FIG. 4A-B is a flowchart describing a method for making a trafficregulation violation decision regarding vehicle activity during a stoplight.

FIG. 5A is a first illustrative image showing a candidate violatingvehicle as it enters a signal-controlled intersection.

FIG. 5B is a second illustrative image showing the candidate violatingvehicle of FIG. 5A running a red light.

FIG. 6A is a first illustrative image showing feature descriptorsextracted from the image in FIG. 5A.

FIG. 6B is a second illustrative image showing feature descriptorsextracted from the image in FIG. 5B.

FIG. 7 is an illustrative image showing correspondences between matchedpairs of feature descriptors extracted from the local neighborhood ofeach feature point in the images in FIGS. 6A and 6B.

FIG. 8A is an illustrative first evidentiary image showing a samplefirst region of interest defined before a stop line.

FIG. 8B is an illustrative second evidentiary image showing a samplesecond region of interest defined after an intersection.

FIG. 8C is an illustrative image showing a remaining cluster of matchingcorrespondences after discarding spurious matches/correspondences/pairs,which do not start within the first ROI_(B) in FIG. 8A and end withinthe second ROI_(A) in FIG. 8B

DETAILED DESCRIPTION

The present disclosure relates to a vision-based system and method fordiscriminating between violating vehicles that run a stop signal andnon-violating vehicles that abruptly stop. Mainly, any vehicle thattriggers the sensors of the RLCS to signal the camera to captureevidentiary images of it is treated by the system as a candidateviolator. The present system automatically processes evidentiary imagesto make a violation decision regarding the candidate violating vehicle.

An overview of the present method 200 is shown in FIG. 2. The methodstarts at S202. The system acquires evidentiary images at S204 capturinga first region of interest before an intersection and a second region ofinterest after the intersection. In FIG. 2, a first illustrative image203A shows a candidate violating vehicle as it enters the intersectionand a second illustrative image 203B shows the vehicle as it proceedsthrough the intersection, where the traffic is being guided using a stopsignal. For illustrative purposes, the term “stop signal” herein mainlyrefers to a traffic light, such as a conventional red light havingmultiple signal lamps each displaying a different color for notifyingdrivers when to stop, use caution, and go. There is furthermore, nolimitation made to the term “intersection”, as the embodiments disclosedherein are amenable to any application making a violation decision whena vehicle proceeds to travel through a regulated region of interest,such as a yield-controlled, stop-controlled, or signal-controlledintersection, all of which are generally regulated to reduce a risk ofvehicular accidents, etc. Generally, the two evidentiary images aretaken in relation to when a vehicle triggers a sensor embedded in theroad, and they assume to capture the vehicle just before an intersectionand at the intersection. Therefore, the sensor can include an inductionloop that, in response to being triggered, delays the camera apredetermined time to enable the capture of the violating vehicleproximate the intersection. The evidentiary images may be taken asadditional evidence in support of a violation discrimination based oninduction loops signals. Alternatively, the evidentiary images make betaken for the express purpose of red light violation discrimination, orfor other purposes. The evidentiary images may be taken with a cameradesigned for still image capture, or the images could be isolated imageframes from a temporal video sequence.

The system extracts a set of feature points and feature descriptors fromeach evidentiary image at S206. The system searches between the twoevidentiary images for pairs of matching feature descriptors at S208.For each pair of matching descriptors, the system extracts anattribute(s) describing a correspondence between the pair at S210. Inone example, the attribute is a computed length of a line connecting thefeature points corresponding to the matching pair of descriptors on theimage plane. In another example, the attribute is a computed anglebetween the line and a second line coinciding with the road direction.

In another example, the system can define a first region of interest(ROI) in the first image located before the stop area and a second ROIin the second image located after the stop area. The system candetermine if a matching point in each pair falls within the first andsecond ROIs. If a cluster of matched feature pairs fall within the firstand second ROIs, then the cluster can correspond to a vehicle travelingalong the road direction. In this embodiment, the attribute is thenumber of matched feature pairs in the cluster.

The system makes a violation decision based on one or a combination ofthe attributes of the matched pair of feature descriptors at S212.Generally, the attribute can be compared to a predetermined threshold,and the violation decision can be based on an outcome of the threshold.The method ends at S214.

FIG. 3 is a schematic illustration of a vision-based system 100 formaking a traffic regulation violation decision regarding a vehiclerunning a stop light. The system 100 includes a traffic regulationenforcement device 102, an image capture device 104—which may beincorporated in a conventional RLCS—linked together by communicationlinks, referred to herein as a network. In one embodiment, the system100 may be in further communication with a user device 106. Thesecomponents are described in greater detail below.

The traffic regulation enforcement device 102 illustrated in FIG. 3includes a controller 110 that is part of or associated with the device102. The exemplary controller 110 is adapted for controlling an analysisof image data received by the system 100. The controller 110 includes aprocessor 112, which controls the overall operation of the device 102 byexecution of processing instructions that are stored in memory 114connected to the processor 112.

The memory 114 may represent any type of tangible computer readablemedium such as random access memory (RAM), read only memory (ROM),magnetic disk or tape, optical disk, flash memory, or holographicmemory. In one embodiment, the memory 114 comprises a combination ofrandom access memory and read only memory. The digital processor 112 canbe variously embodied, such as by a single-core processor, a dual-coreprocessor (or more generally by a multiple-core processor), a digitalprocessor and cooperating math coprocessor, a digital controller, or thelike. The digital processor, in addition to controlling the operation ofthe device 102, executes instructions stored in memory 114 forperforming the parts of the method outlined in FIGS. 2 and 4. In someembodiments, the processor 112 and memory 114 may be combined in asingle chip.

The device 102 may be embodied in a networked device, such as the imagecapture device 104, although it is also contemplated that the device 102may be located elsewhere on a network to which the system 100 isconnected, such as on a central server, a networked computer, or thelike, or distributed throughout the network or otherwise accessiblethereto. In other words, the processing can be performed within theimage capture device 104 on site or in a central processing offline orserver computer after transferring the evidentiary images through anetwork. In one embodiment, the image source 104 can be a device adaptedto relay and/or transmit the images to the device 102. In anotherembodiment, the image data 130 may be input from any suitable source,such as a workstation, a database, a memory storage device, such as adisk, or the like. The image source 104 is in communication with thecontroller 110 containing the processor 112 and memories 114.

The stages disclosed herein are performed by the processor 112 accordingto the instructions contained in the memory 114. In particular, thememory 114 stores an image buffering module 116, which receivesevidentiary photographs (“images”) capturing a first area before anintersection and a second area after an intersection; a correspondencedetermination module 118, which extracts feature points in each image,computes feature descriptors for each extracted feature point, anddetermines a correspondence between pairs of feature points havingmatching feature descriptors in the at least two evidentiary images; anattribute generation module 120, which extracts attribute(s) fordescribing each correspondence; a violation determination module 122,which determines if the candidate violating vehicle is in violation ofrunning the intersection in lieu of a stop light using the extractedattribute(s); and, a violation notification module 124, which notifies auser of the violation decision. Embodiments are contemplated whereinthese instructions can be stored in a single module or as multiplemodules embodied in different devices. The modules 116-124 will be laterdescribed with reference to the exemplary method.

The software modules as used herein, are intended to encompass anycollection or set of instructions executable by the device 102 or otherdigital system so as to configure the computer or other digital systemto perform the task that is the intent of the software. The term“software” as used herein is intended to encompass such instructionsstored in storage medium such as RAM, a hard disk, optical disk, or soforth, and is also intended to encompass so-called “firmware” that issoftware stored on a ROM or so forth. Such software may be organized invarious ways, and may include software components organized aslibraries, internet-based programs stored on a remote server or soforth, source code, interpretive code, object code, directly executablecode, and so forth. It is contemplated that the software may invokesystem-level code or calls to other software residing on a server (notshown) or other location to perform certain functions. The variouscomponents of the device 102 may be all connected by a bus 126.

With continued reference to FIG. 3, the device 102 also includes one ormore communication interfaces 128, such as network interfaces, forcommunicating with external devices. The communication interfaces 128may include, for example, a modem, a router, a cable, and and/orEthernet port, etc. The communication interfaces 128 are adapted toreceive the images (“image data 130”) as input.

The device 102 may include one or more special purpose or generalpurpose computing devices, such as a server computer, controller, or anyother computing device capable of executing instructions for performingthe exemplary method.

FIG. 3 further illustrates the device 102 connected to an image capturedevice 104 for acquiring and/or providing the image data in electronicformat. The image capture device 104 (hereinafter “camera 104”) mayinclude one or more surveillance cameras that capture photographs fromthe scene of interest. The number of cameras may vary depending on alength and location of the area being monitored. It is contemplated thatthe combined field of view of multiple cameras typically comprehends theentire area surrounding the intersection at least in the road direction.For performing the method at night in areas without external sources ofillumination, the camera 104 can include near infrared (NIR)capabilities. In the contemplated embodiment, the camera 104 is a highresolution camera to enable the identification of violating vehiclesthrough processes such as automatic license plate recognition (ALPR),etc.

With continued reference to FIG. 3, the image data 130 undergoesprocessing by the traffic regulation enforcement device 102 to output aviolation decision 132.

Furthermore, the system 100 can display the violation decision and/oroutput in a suitable form on a graphic user interface (GUI) 134. The GUI134 can include a display for displaying the information, to users, anda user input device, such as a keyboard or touch or writable screen, forreceiving instructions as input, and/or a cursor control device, such asa mouse, touchpad, trackball, or the like, for communicating user inputinformation and command selections to the processor 112. Alternatively,the device 102 can provide the violation decision to the output device106, which can display the decision to a user, such as a trafficenforcement officer, or a notification 136 to the authority in charge ofissuing citations. Furthermore, in one contemplated embodiment,violation decision can be transmitted to another computer application,which can perform additional processing on the image to identify ownersof any violating vehicle for the purpose of issuing citations.

FIG. 4A-B is a flowchart describing a method 400 for making a trafficregulation violation decision regarding vehicle activity during a stoplight signal. The method starts at S402.

At S404, the image buffering module 116 acquires evidentiary imagescaptured from the RLCS. As mentioned supra, the RLCS operates to captureimages of a vehicle during a stop light. The RLCS employs an in-groundinduction loop having sensors that trigger a camera to capture images ofvehicle activity in the scene of interest. However, embodiments arecontemplated whereby the module 116 acquires the evidentiary images froma camera 104, which is in communication with a traffic light controllersuch that it captures images of the scene (e.g., intersection) ofinterest during the periods that traffic light is red. In yet anotherembodiment, the module 116 can acquire images from a camera that isoperating to capture the images in response to user input and/orinstruction. Generally, the module 116 acquires two evidentiary imagesfor processing: a first image of the intersection before a stop line (orcrossroad) and a second image of an area located within/after theintersection. Embodiments are contemplated, however, which process morethan two images where the scene of interest is on the image plane.Generally, these images are assumed to capture the activity of acandidate violating vehicle. FIG. 5A is a first illustrative imageshowing a candidate violating vehicle 50 as it enters asignal-controlled intersection 52. FIG. 5B is a second illustrativeimage showing the candidate violating vehicle 50 of FIG. 5A running ared light 54.

At S406, the correspondence determination module 118 extracts a set offeature points from each evidentiary image. In one embodiment, themodule can extract scale invariant feature points for employing infurther processing. One approach for extracting scale invariant featuretransform (SIFT) feature points is provided by David G. Lowe, et al., inthe publication titled Distinctive Image Features from Scale-InvariantKeypoints, in International Journal of Computer Vision 60.2 (2004) atpg. 91-110 and is fully incorporated herein. One approach for extractingspeeded-up robust feature points (SURF) is provided by Herbert Bay, etal., in the publication titled Speeded-up Robust Features (SURF), inComputer Vision and Image Understanding 100.3 (2008) at 346-359 and isfully incorporated herein. One approach for extracting Harris cornerfeatures is provided by Chris Harris and Mike Stephens in thepublication titled A Combined Corner and Edge Detector” in Alvey VisionConference, Vol. 15, 1998 and is fully incorporated herein. One approachis contemplated for performing a fast accelerated segment test (FAST),which is described by Edward Rosten and Tom Drummond in the publicationtitled Machine Learning For High-Speed Corner Detection in ComputerVision-ECCV 2006 and is fully incorporated herein. One approach forextracting minimum eigenvalue algorithm feature points is provided byCarlo Tomasi and Takeo Kanade in the publication titled Detection andTracking of Point Features, in School of Computer Science, CarnegieMellon Univ. 1991 and is fully incorporated herein. Another approach forextracting maximally stable extremal region (MSER) feature points isprovided by P-E Forssen and David G. Low in the publication titled ShapeDescriptors for Maximally Stable Extremal Regions in Computer Vision2007 and is fully incorporated herein.

FIG. 6A is a first illustrative image showing (SURF) feature pointsextracted from the image in FIG. 5A. Similarly, FIG. 6B is a secondillustrative image showing (SURF) feature points extracted from theimage in FIG. 5B.

The module 118 computes feature descriptors for each of the extractedfeature points at S408. Mainly, the feature descriptors are typicallycomputed in the local neighborhood of each feature point. There is nolimitation made herein for which process is used for computing thefeature descriptors. Rather, any known approach can be applied. Oneapproach for computing feature descriptors using a histogram ofgradients (HOG) is described by Navneet Dalal and Bill Triggs in thepublication titled Histograms of Oriented Gradients for Human Detectionin Computer Vision and Pattern Recognition 2005, which is incorporatedherein fully. Another approach for computing feature descriptors usingdifferences of Gaussian (DoG) filters is provided in the publicationDistinctive Image Features from Scale-Invariant Points. Another approachfor computing feature descriptors using Haar-wavelet responses isprovided in the publication Speeded-Up Robust Features (SURF).

Furthermore, the feature descriptors can be calculated in the localneighborhood of each feature point at different scales, particularlybecause certain features—such as SURF features—are scale invariant. Thedimensionality of feature descriptors varies depending on the processused for extracting the features. For example, the dimensionality of theSURF descriptors is lower than the dimensionality of SIFT descriptors.Because the processing time/speed in the next stage of the operationdepends on the dimension and complexity of the feature descriptor, whichcan be traded-off to keep a balance with the distinctiveness of thedescriptor, a process can be performed on each feature descriptor toreduce its dimension. One approach to reduce dimension using principalcomponent analysis (PCA-SIFT) is proposed by Yan Ke and Rahul Sukthankarin the publication titled PCA-SIFT: A More Distinctive RepresentationFor Local Image Descriptors, in Computer Vision and Pattern Recognition2004, which is incorporated fully herein.

Returning to FIG. 4, the computed feature descriptors in the firstevidentiary image are compared against the computed feature descriptorsin the second evidentiary image to find matching pairs of featuredescriptors. In other words, a correspondence is determined betweenfeature points having matching feature descriptors at differentlocations in the at least two evidentiary images at S410. Becauselocations of matched features can be the same in both of the evidentiaryimages, which is particularly expected for stationary objects, no actionof interest is observed in these regions. However, a correspondencebetween a pair of feature points in different locations in the twoimages, but having matching feature descriptors, can be determined basedon a distance between the descriptor points on the image plane. There isno limitation made herein to the distance metric used to determine thedistance. Example approaches include a sum of squared distances (SSD), asum of absolute distances, Mahalanobis and Euclidian distance, etc. Thecalculated distance can be compared with a predetermined threshold toidentify the matched features between different images. FIG. 7 is anillustrative image showing correspondences between matched pairs offeature descriptors extracted from the local neighborhood of eachfeature point in the images in FIGS. 6A and 68. FIG. 7 represents anoverlap of the first and second images, where the position of candidateviolating vehicle 70 in the first image is shown in phantom.

Similar feature descriptors can be extracted/calculated for a number offeature points between the two evidentiary images that are associatedwith different objects. FIG. 7 illustrates this scenario with a numberof spurious correspondences (such as, for example, path 72) between thematched points (for example, 74 and 76).

To remove the spurious matches, the attribute generation module 120searches for a number of matched feature pairs—making up a coherentcluster—that each start and end in defined regions of interests. Tocompute the number of matched feature pairs, the attribute generationmodule 120 first defines a first region of interest (ROI) located on theimage plane before the intersection in a first one of the evidentiaryimages at S412. For example, the first ROI can be defined before anexisting or virtual stop bar. When the RLCS camera(s) takes the firstevidentiary image, it aims to capture the candidate violating vehiclebefore the intersection. FIG. 8A is an illustrative image showing asample first ROI 82 defined before a stop line 84. A candidate violatingvehicle 86 is captured in the evidentiary image. The module 120determines if any of the feature points of the matched feature pairsfall within the first ROI in the first evidentiary image at S414. Thisdetermination is made by identifying a location of the feature pointcorresponding to a first one in each pair of matching featuredescriptors. Then, the module 120 determines if that location fallsinside or outside the first ROI. In response to the location fallingoutside the first ROI (NO at S414), the module 120 associates thecorrespondence as belonging to a spurious match and discards thecorrespondence at S416. FIG. 8C is an illustrative image showing theremaining matched (SURF) features after eliminating the spuriousmatches/correspondences/pairs, which do not start within the first ROI.

In response to the location falling inside the first ROI (YES at S414),the module 120 defines a second region of interest located on the imageplane after the intersection in a second one of the evidentiary imagesat S418. For example, the second ROI can be defined at and/or after theintersection. When the RLCS camera(s) takes the second evidentiaryimage, it aims to capture violating vehicles running through theintersection. FIG. 8B is an illustrative second evidentiary imageshowing a sample second ROI 88 defined after the intersection 90. Thecandidate violating vehicle 86 is captured in the evidentiary image. Themodule 120 determines if any of the feature points of the matchedfeature pairs fall within the second ROI in the second evidentiaryimage. This determination is made by identifying a location of thefeature point corresponding to a second one in each pair of matchingfeature descriptors. Then, the module 120 determines if that locationfalls inside or outside the second ROI at S420. In response to thelocation falling outside the second ROI (NO at S420), the module 120associates the correspondence as belonging to a spurious match anddiscards the correspondence at S416. FIG. 8C also shows the remainingmatched features after eliminating the spuriousmatches/correspondences/pairs, which do not end within the secondROI_(A).

In response to the location falling inside the second ROI (YES at S420),the module 120 performs further thresholding operation(s) on theremaining correspondences to discriminate between spurious and truematches. However, the aforementioned processes at S412-S420 ofdetermining whether the feature points fall outside first and secondROIs, for purposes of discarding spurious matches, can be omitted incontemplated embodiments. Particularly, the spurious matches may also,in certain embodiments, be identified using the thresholding operationson each correspondence verses just the remaining correspondences.

Continuing with FIG. 4B, the attribute generation module 120 extracts atleast one attribute describing each correspondence at S422. In oneembodiment, the attribute includes a distance between (i.e., a length ofa line connecting) locations of the matching pair feature points on theimage plane of the two evidentiary images. The module 120 computes thelength L of the line at S424 using the equation:L=√{square root over ((x ₂ −x ₁)²+(y ₂ −y ₁)²)},  (1)where (x₁,y₁) is a location of a feature point in a first image and(x₂,y₂) is a location of a matching feature point in a second image.

In another embodiment, the attribute includes an angle θ formed betweena first line extending between locations of the matching feature pointsin the at least two evidentiary images and the second line beingcoincident along a road direction that the candidate violating vehicleis expected to travel through the intersection. The module 120 computesthe angle θ at S426 using the equation:

$\begin{matrix}{{\theta = {{atan}\left( \frac{y_{2} - y_{1}}{x_{2} - x_{1}} \right)}},} & (2)\end{matrix}$where (x₁,y₁) is a location of a feature point in a first image and(x₂,y₂) is a location of a matching feature point in a second image.

After the attribute generation module 120 computes at least oneattribute describing each correspondence, the violation determinationmodule 122 determines whether the candidate violating vehicle isviolating a traffic enforcement regulation at S428 by running throughthe yield-controlled, stop-controlled, or signal-controlledintersection. The module 122 makes a violation/non-violation decisionbased on the computed attribute(s).

As part of this decision, the module 122 identifies if there is aviolating vehicle in the scene. In the discussed embodiment, a violationcan only occur if the matched features on the violating vehicle fall onthe road direction both before and after the intersection, althoughother embodiments are contemplated to consider left hand and right handturns in violation of the traffic regulation. In these alternativeembodiments, the second ROI can be defined past the intersection in thecross lane. In the discussed embodiment, the violation decision can bebased on a criterion of finding coherent cluster of matched featuresthat comply with at least one predetermined threshold.

As part of this violation decision in one embodiment, the module 122identified the matched feature pairs that start within the first ROI_(B)and end within the second ROI_(B) at S414, S420. Among these pairs, themodule 122 searches for a coherent cluster of correspondences. FIG. 8Cshows an example cluster 92 of correspondences traveling together in theroad direction. This cluster 92 corresponds to a number of matched pairsof vehicle features for vehicle 86. In one embodiment, the module 122can determine if a group of correspondences belongs to a cluster if thenumber of matched pairs meets a predetermined number threshold.

In response to a cluster of correspondences satisfying the condition forstarting and ending within the defined first and second ROIs, the module122 can determine if the attribute associated with at least onecorrespondence in the cluster meets a predetermined threshold.

In the illustrative embodiment where the attribute is the distance Lbetween (i.e., a length of a line connecting) locations of the matchingpair feature points on the image plane of the two evidentiary images,the distance can be compared to a predetermined length threshold atS430. In one embodiment, the threshold can be the distance between thestop line and the other side of the intersection. In another embodiment,the threshold can be the distance between the stop line and the middleof the intersection, where the system can presume that a vehiclecaptured in the middle of the intersection will proceed to travelthrough the intersection. Although, any length threshold can be used toidentify violators.

In one embodiment, the threshold can be zero “0”. Locations are the samefor most feature points having matching feature descriptors in theevidentiary images, particularly because of the stationary objects inthe scene. A violating vehicle, however, will be located at differentplaces in the two evidentiary images (see FIGS. 5A and 58) because it ismoving through the intersection when the two images are captured. Thedistance between the pair of feature points of moving objects isdifferent than the distance between stationary objects, which isexpected to be zero “0”. Accordingly, the computed length is compared toa predetermined threshold of zero “0” to eliminate the correspondencesbetween stationary objects.

In response to the computed length not meeting the predeterminedthreshold (NO at S430), the module 122 can classify the candidateviolating vehicle as belonging to a non-violating vehicle at S432. Inresponse to the computed length meeting and exceeding the predeterminedthreshold (YES at S430), the module 122 can classify the candidateviolating vehicle as being a violating vehicle at S434.

However, another embodiment is contemplated where in response to thecomputed length meeting and exceeding the predetermined threshold, themodule 122 can perform at least a second thresholding operation on theselect correspondence using a different attribute. This embodimentanticipates at least an additional attribute can be computed for thecorrespondence and compared to a second threshold.

In the illustrative embodiment where the attribute is the angle θ formedbetween the first line extending between locations of the matchingfeature points in the at least two evidentiary images and the secondline being coincident along a road direction that the candidateviolating vehicle is expected to travel through the intersection, theviolating vehicle is expected to move along the road direction.Accordingly, the disclosure anticipates that a correspondence (or line)connecting a pair of feature points of the matched features also extendsalong the road direction. Therefore, the computed angle θ is compared toa predetermined threshold to eliminate the correspondences betweenspurious objects and/or moving objects (e.g., birds, pedestrians walkingin the background, etc.) traveling in different directions that do notcomply with the road direction at S430. In one embodiment, the thresholdcan defined as a predetermined interval around a calculated roadangle—i.e., road direction on the image plane. In the discussed example,the road predetermined threshold can be (±5°), however a different anglethreshold can be used to identify violators. For example, in theillustrative figures discussed herein, the road direction is in astraight line, but embodiments are contemplated where the intersectionincludes a number of road segments (e.g., 3-way intersection, 5-wayintersection, etc.) where the crossing roads/streets are notperpendicular to each other or where the first side of the street in theroad direction located before the intersection is not perpendicular tothe second side of the street in the road direction located after theintersection.

In response to the computed angle θ meeting and exceeding thepredetermined threshold (YES at S430), the violation determinationmodule 122 can classify the candidate violating vehicle as belonging toa violating vehicle at S434. In response to the computed angle θ notmeeting the predetermined threshold (NO at S430), the module 122 canclassify the candidate violating vehicle as belonging to a non-violatingvehicle at S432. The method ends at S436.

In an alternate embodiment, after the matched features are extractedbetween the evidentiary images, the system can apply the sets offeatures to a linear/non-linear classifier (e.g., SVM), which is trainedbeforehand using a classical supervised machine learning approach. Inthis approach, a vector of attributes is calculated for each of thematched feature pair. The classifier then makes a decision based on thevector of the attributes of the matched features.

One aspect of the present disclosure is a reduction in the number offalse positives that generally result from the conventional RLCSenforcement system. Another aspect of the present disclosure is that thesystem and method employ the existing infrastructure of existing RLCSenforcement systems and can be easily integrated into those existingsystem.

Although the method 200, 400 is illustrated and described above in theform of a series of acts or events, it will be appreciated that thevarious methods or processes of the present disclosure are not limitedby the illustrated ordering of such acts or events. In this regard,except as specifically provided hereinafter, some acts or events mayoccur in different order and/or concurrently with other acts or eventsapart from those illustrated and described herein in accordance with thedisclosure. It is further noted that not all illustrated steps may berequired to implement a process or method in accordance with the presentdisclosure, and one or more such acts may be combined. The illustratedmethods and other methods of the disclosure may be implemented inhardware, software, or combinations thereof, in order to provide thecontrol functionality described herein, and may be employed in anysystem including but not limited to the above illustrated system 100,wherein the disclosure is not limited to the specific applications andembodiments illustrated and described herein.

It will be appreciated that variants of the above-disclosed and otherfeatures and functions, or alternatives thereof, may be combined intomany other different systems or applications. Various presentlyunforeseen or unanticipated alternatives, modifications, variations orimprovements therein may be subsequently made by those skilled in theart which are also intended to be encompassed by the following claims.

What is claimed is:
 1. A method to detect a vehicle running a stopsignal, the method comprising: acquiring at least two evidentiary imagesof a candidate violating vehicle captured from at least one cameramonitoring an intersection, the at least two evidentiary imagesincluding a first image of an intersection before a stop line and asecond image of an area located within or after the intersection;extracting feature points in each of the at least two evidentiaryimages; computing feature descriptors for each of the extracted featurepoints; determining a correspondence between feature points havingmatching feature descriptors at different locations in the at least twoevidentiary images; extracting at least one attribute for eachcorrespondence by determining if the each correspondence belongs to oneof a spurious match and a true match; for each correspondence belongingto a true match, comparing the attribute to a threshold; and,classifying the candidate violating vehicle as belonging: to one of aviolating vehicle running the stop signal and a non-violating vehiclebased on the comparing.
 2. The method of claim 1, wherein theclassifying includes: in response to the angle meeting and exceeding thepredetermined threshold, classifying the candidate violating vehicle asbelonging to a non-violating vehicle, and in response to the angle notmeeting the predetermined threshold, classifying the candidate violatingvehicle as belonging to a violating vehicle.
 3. The method of claim 1,wherein the attribute includes a length of a line connecting locationsof the matching feature points in the at least two evidentiary imagesand the threshold includes a predetermined length.
 4. The method ofclaim 3, wherein the length of the line is computed using the equationL=√{square root over ((x₂−x₁)²+(y₂−y₁)²)}, where (x₁, y₁) is a locationof a feature point in a first image and (x₂, y₂) is a location of amatching feature point in a second image.
 5. The method of claim 3,further comprising: in response to the length meeting and exceeding thepredetermined threshold, classifying the candidate violating vehicle asbelonging to a violating vehicle, and in response to the length notmeeting the predetermined threshold, classifying the candidate violatingvehicle as belonging to a non-violating vehicle.
 6. The method of claim1, where the attribute includes an angle formed between a first line anda second line and the threshold includes a predetermined angle, thefirst line extending between locations of the matching feature points inthe at least two evidentiary images and the second line being coincidentalong a road direction that the candidate violating vehicle is expectedto travel through the intersection.
 7. The method of claim 5, whereinthe angle of the line is computed using the equation${\theta = {{atan}\left( \frac{y_{2} - y_{1}}{x_{2} - x_{1}} \right)}};$wherein (x₁, y₁) is a location of a feature point in a first image and(x₂, y₂) is a location of a matching feature point in a second image. 8.The method of claim 1 further comprising: defining a first region ofinterest (ROI) located before the intersection in a first of the atleast two images; determining a first location of a first one in a pairof matching points in the first image; and, in response to the locationfalling outside the first ROI, associating the correspondence as aspurious match.
 9. The method of claim 8 further comprising: defining asecond ROI located after the intersection in a second of the at leasttwo images; determining a second location of a second one in the pair ofmatching feature points in the second image; and in response to thesecond location falling outside the second ROI, associating thecorrespondence as a spurious match.
 10. The method of claim 9 whereinthe extracting at least one attribute for each correspondence includes:extracting the at least one attribute for each correspondence notbelonging to a spurious match and discarding the each correspondencebelonging to a spurious match.
 11. A system for detecting a vehiclerunning a stop signal, the system comprising a traffic regulationenforcement device including a memory and a processor in communicationwith the processor configured to: acquire at least two evidentiaryimages of a candidate violating vehicle captured from at least onecamera monitoring an intersection, the at least two evidentiary imagesincluding a first image of an intersection before a stop line and asecond image of an area located within or after the intersection;extract feature points in each of the at least two evidentiary images;compute feature descriptors for each of the extracted feature points;determine a correspondence between feature points having matchingfeature descriptors at different locations in the at least twoevidentiary images; extract at least one attribute for eachcorrespondence by determining if the each correspondence belongs to oneof a spurious match and a true match; for each correspondence belongingto a true match, comparing the attribute to a threshold; and,classifying the candidate violating vehicle as belonging to one of aviolating vehicle running the stop signal and a non-violating vehiclebased on the comparing.
 12. The system of claim 11, wherein theattribute includes a length of a line connecting locations of thematching feature points in the at least two evidentiary images and thethreshold includes a predetermined length.
 13. The system of claim 12,wherein the length of the line is computed using the equation L=√{squareroot over ((x₂−x₁)²+(y₂−y₁)²)}, where (x₁, y₁) is a location of afeature point in a first image and (x₂, y₂) is a location of a matchingfeature point in a second image.
 14. The system of claim 12, wherein theprocessor is further configured to: in response to the length meetingand exceeding the predetermined threshold, classify the candidateviolating vehicle as belonging to a violating vehicle, and in responseto the length not meeting the predetermined threshold, classify thecandidate violating vehicle as belonging to a non-violating vehicle. 15.The system of claim 11, where the attribute includes an angle formedbetween a first line and a second line and the threshold includes apredetermined angle, the first line extending between locations of thematching feature points in the at least two evidentiary images and thesecond line being coincident along a road direction that the candidateviolating vehicle is expected to travel through the intersection. 16.The system of claim 15, wherein the angle of the line is computed usingthe equation${\theta = {{atan}\left( \frac{y_{2} - y_{1}}{x_{2} - x_{1}} \right)}};$wherein (x₁, y₁) is a location of a feature point in a first image and(x₂, y₂) is a location of a matching feature point in a second image.17. The system of claim 15, wherein the processor is further configuredto: in response to the angle meeting and exceeding the predeterminedthreshold, classify the candidate violating vehicle as belonging to anon-violating vehicle, and in response to the angle not meeting thepredetermined threshold, classify the candidate violating vehicle asbelonging to a violating vehicle.
 18. The system of claim 11, whereinthe processor is further configured to: define a first region ofinterest (ROI) located before the intersection in a first of the atleast two images; determine a first location of a first one in a pair ofmatching feature points in the first image; and, in response to thelocation falling outside the first ROI, associate the correspondence asbeing a spurious match.
 19. The system of claim 18, wherein theprocessor is further configured to: define a second ROI located afterthe intersection in a second of the at least two images; determine asecond location of a second one in a pair of matching feature points inthe second image; and in response to the second location falling outsidethe second ROI, associate the correspondence as being a spurious match.20. The system of claim 19, wherein the processor is configured to:extract the at least one attribute for each correspondence not belongingto a spurious match and discard the each correspondence belonging to aspurious match.